Video Extrapolation in Space and Time

Novel view synthesis (NVS) and video prediction (VP) are typically considered disjoint tasks in computer vision. However, they can both be seen as ways to observe the spatial-temporal world: NVS aims synthesize a scene from new point of view, while VP see time. These two provide complementary signals obtain representation, viewpoint changes spatial observations inform depth, temporal motion cameras individual objects. Inspired by these observations, we propose study problem Video Extrapolation Space Time (VEST). We model that leverages self-supervision cues tasks, existing methods only solve one them. Experiments show our method achieves performance better than or comparable several state-of-the-art on indoor outdoor real-world datasets. (Project page: https://cs.stanford.edu/~yzzhang/projects/vest/ .)